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Abstract 

Automated planning and scheduling, including au- 
tomated path planning, has been integrated with an 
Internet-based distributed operations system for plan- 
etary rover operations. The resulting prototype sys- 
tem enables faster generation of valid rover command 
sequences by a distributed planetary rover operations 
team . The Web Interface for Telescience (WITS) pro- 
vides Internet-based distributed collaboration, the Au- 
tomated Scheduling and Planning ENvironment (AS- 
PEN) provides automated planning and scheduling, 
and an automated path planner provides path plan- 
ning. The system was demonstrated on the Rocky7 
research rover at JPL. 

1 Introduction 

Planetary mission operations consists of receiving 
downlink data, processing the data, generating the 
next command sequence, and then uplinking the se- 
quence to the spacecraft. In past missions, this pro- 
cess for generating each command sequence could take 
days or weeks and require numerous operations per- 
sonnel. The highly successful 1997 Mars Pathfinder 
mission was able to reduce the time for generating a 
command sequence to the Sojourner rover to about 12 
hours. New distributed operations, automated plan- 
ning and scheduling, and automated path planning 
technologies described in this paper are being devel- 
oped to reduce this operations planning time further 
for future Mars rover missions. 

For the Pathfinder Sojourner rover daily mission 
operations, the engineering team determined the state 
of the rover and provided it to the science team. The 
science team, co-located at JPL with the engineer- 
ing team, then collaborated to specify a high-level 
science plan consisting of a list of science activities 
which they wanted to have included in the next up- 
link rover command sequence. This list of science ac- 
tivities included science activity parameters and lo- 


cations, but the trade-offs for including the activities 
versus engineering constraints were not known. The 
high-level science plan was given to the engineering 
team, which had the responsibility for generating the 
low-level command sequence that satisfied the science 
plan as well as possible while considering operations 
constraints, resource constraints, and hardware con- 
straints for the rover. The process of generating the 
command sequence was largely manual relying on the 
significant skills of the operations personnel. If there 
was time after the engineering team produced a se- 
quence, then the science team reviewed it in order to 
verify that desired science activities were included in 
the final sequence, and iterated with the engineering 
team until a sequence was generated which satisfied 
both science and engineering constraints. The engi- 
neering team was usually able to accommodate most 
of the science objectives, but usually there was not 
enough time for iterations on the sequence with the 
science team. 

Since the Pathfinder mission was relatively short, a 
few months long, the science team could be co-located 
with the engineering team at JPL. Also, the operations 
team members were able to work very long hours co- 
incident with Mars daily cycles — meaning that they 
might work 12, and often more, hours in a day starting 
in the middle of the Earth night. 

Future Mars rover missions, i.e., in 2001, 2003, and 
2005, will be more challenging by being longer, with 
missions lasting up to a year or longer, with more so- 
phisticated rovers and increased numbers of science 
instruments, and with more constrained communica- 
tions opportunities. New technologies need to be pro- 
vided for mission operations to maximize mission sci- 
ence return while minimizing operations costs and pro- 
viding a reasonable working environment for the oper- 
ations personnel for the long duration of the missions. 

This paper describes the results of recent work to 
integrate Internet-based distributed operations with 
automated resource analysis and scheduling of activ- 
ities to enable increased science return at lower cost 



for fu ( ure planetary mvfr missions. Automated path 
planning has also hern added to t he system so that the 
automated scheduler can determine traverse costs be- 
tween science targets when performing the scheduling. 
The system takes science activity requests specified by 
the distributed operations team and generates a valid 
command sequence which can he sent to a rover. The 
integrated system was used to command the Rocky7 
research rover [1] at JPL. 

The integrated prototype system of this paper has 
many of the basic features needed in future planetary 
rover mission operations. Distributed operations en- 
ables full participation by scientists from their home 
institutions, as well as collaborative planning by the 
science and engineering teams. Automated resource 
analysis and scheduling enables the science team to 
perform trade-offs on science objectives and generate 
a sequence which they have confidence is likely to be 
achievable within mission resource constraints, before 
handing it off to the engineering team. Automated 
path planning is needed by the resource analysis sys- 
tem to determine traverse costs between science tar- 
gets. Also, it enables the science team to generate 
initial approximate paths without relying on a plan- 
ning iteration with the engineering team who have the 
skill to do it manually. Mission planning time will be 
reduced by having automated, fast, constraints check- 
ing and valid sequence generation. The number of 
necessary iterations between the science and engineer- 
ing teams will decrease by providing more operations 
constraints checking capability to the science team via 
the automated scheduler and path planner. The ex- 
pertise of individuals in the operations team will still 
be required to generate and validate a final safe com- 
mand sequence, but the sequence that they get from 
the science team will be much more complete and vi- 
able. 

2 System Architecture 

The system architecture is shown in Figure 1. The 
architecture is a simplified diagram of planetary rover 
mission operations which describes the interaction of 
the technologies of this paper. The Downlink sys- 
tem receives downlink data from the rover and the 
Data Processing system processes the data and puts 
both the raw data and data products, such as im- 
ages and range maps, into the Database. Distributed 
Sequence Generation provides collaborative sequence 
generation by the geographically distributed science 
and engineering teams and is performed here by the 
Web Interface for Telescience (WITS), described be- 
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Figure 1: Operations System Architecture 


low in Section 3. WITS provides distributed visual- 
ization and activity specification. The Internet-based 
WITS client communicates with a server system at 
JPL which includes other elements of the architec- 
ture. The Automated Planning and Scheduling sys- 
tem receives sequences and performs resource analysis 
on them and generates output sequences which are 
valid within modeled operations constraints. The Au- 
tomated Scheduling and Planning ENvironment (AS- 
PEN), described in Section 4, was used as the Auto- 
mated Planning and Scheduling system here. Various 
other planning tools might be used to support dis- 
tributed sequence generation or automated planning 
and scheduling such as automated path planning or 
automated computation of planetary ephemeris data. 
A new automated path planning system, described in 
Section 5, was used for path planning. The SPICE 
planetary ephemeris system [2] was used to automat- 
ically compute the angle to the sun from the rover 
position. 

The Sequence Finalization system would be respon- 
sible for taking the sequence generated by the dis- 
tributed operations team and outputting the final, 
valid, command sequence which is uplinked to the 
rover for execution. This role was performed in the 
Pathfinder Sojourner rover mission by the engineering 
team using the Rover Control Workstation [3]- In the 
system described in this paper, the sequence gener- 
ated via the WITS-ASPEN system was valid enough 
to send directly to the rover for execution. 

The various software modules could run on differ- 





r nf c r x 1 1 pi i f rrs. In tin* examples of this paper. three 
computers were used. 'I'li** ASPEN system rail on one 
computer, t in* rest of tin* server systems ran on a sec- 
ond computer, aurl the WITS (‘limit ran on a third 
computer (of course; the WITS client could have been 
accessed from any computer on the Internet). The 
Database was stored in flat files in a structured direc- 
tory hierarchy of a file system. The ASPEN system, 
path planner, and SPICE systems all ran as active 
server systems. The WITS client accessed Database 
data as URL calls through a web server on a Sun Unix 
computer or via CGI programs which read or wrote 
data in the Database. The WITS client communicated 
with the ASPEN, path planner, and SPICE server sys- 
tems via CGI programs which acted as clients to the 
server programs. An Uplink server program took the 
final command sequence from WITS via a CGI pro- 
gram and sent it to the rover for execution. 

3 Distributed Operations 

The Web Interface for Telescience (WITS) [4] is 
an Internet-based tool that enables scientists to col- 
laboratively participate in planetary lander and rover 
missions from their home institutions. WITS enables 
the viewing of downlinked images and results in var- 
ious ways, terrain feature measurement and annota- 
tion, and planning of daily mission activities. WITS 
is written in the Java language and is accessible by 
mission scientists and the general public via a com- 
mercial w r eb browser. The public cam use WITS to 
plan and simulate their own rover missions. WITS 
was used in the 1997 Mars Pathfinder mission for pub- 
lic outreach and limited science operations and will 
be a nominal science operations tool in the 1998 lan- 
der and 2001 and 2003 lander-rover missions to Mars. 
Information on WITS can be found on the Internet 
(http://robotics.jpl.nasa.gov/tasks/wits/). 

Various view windows in WITS permit looking at 
the images and data in different ways. Planning and 
measurement can be done in all views, with all in- 
puts displayed in all views. The Descent View has 
an orbital image or an image taken by the spacecraft 
during descent to the surface. The Panorama View 
shows an overhead view of the area around the lander 
or rover in various ways, e.g., color coded elevation 
map or texture map. The Mosaic View is a mosaic of 
the actual panorama images. For the Pathfinder mis- 
sion, the panoramic images were taken by a camera 
on the lander, as will be the case for the 1998 lander 
mission. For the rover missions, the panoramic im- 
ages will also be taken from a mast, on the rover. The 


Close-up View provides viewing of images taken by the 
navigation cameras on the rover. The 31) View pro- 
vides 3D solid-model visuali/al ion and simulat ion, and 
viewing from different directions. Tin' user can specify 
science targets and waypoints in any view. Activities 
to be performed at a target or waypoint are specified 
by opening a Task window. Access to the mission sys- 
tem is controlled through the use of user logins and 
passwords. 

Distributed collaboration is achieved in WITS via 
sharing a common server-located database. A user can 
save inputs such as science targets and science activi- 
ties to the common database by pressing a save icon. 
All users can see the current shared data by press- 
ing the load icon. All users share a common sequence 
which is collaboratively generated. 

4 Automated Planning and Scheduling 

The Automated Scheduling and Planning ENvi- 
ronment (ASPEN) [5] is an automated planning and 
scheduling tool developed using Artificial Intelligence 
techniques and algorithms. ASPEN generates a se- 
quence which is an ordered list of activities. A model 
of the system to be scheduled is first generated which 
describes the possible set of activities, the constraints 
between those activities, and the resources required 
for each activity. ASPEN can be used in both an in- 
teractive mode where a user inputs and modifies ac- 
tivity requests directly, or in a batch mode where a 
sequence is automatically input, e.g., from WITS as 
used in this paper, and the sequence is automatically 
modified to meet modeled resource constraints. AS- 
PEN uses a client/server design where the server con- 
tains the schedule data and algorithms, and clients, 
such as the ASPEN Graphical User Interface (GUI), 
can send schedule modifications or request informa- 
tion. The GUI displays the activities, their loca- 
tion in time, and the resources and states affected by 
the activities. Conflicts are displayed when any of 
the constraints are violated. The GUI also provides 
an interface for manually editing the schedule when 
used in an interactive mode. More information on 
ASPEN can be found on the Internet (http://www- 
aig.jpl.nasa.gov/planning/aspen/). 

Several algorithms have been implemented for gen- 
erating initial schedules from the requests, and repair- 
ing existing schedules with conflicts. These algorithms 
are generic and can be used on any given model. The 
most commonly used algorithm in ASPEN is based 
on a technique called iterative repair (5, 6, 7]. With 
iterative repair, the scheduler determines the conflicts 



m an input schedule of requests and loops through Jill 
of the mnllirts, trying to resolve each I jy performing 
a sclifM lull* modification. A conflict in tin* schedule 
is simply a violation of one of the resource, state, or 
temporal roiistr. lints defined in I In * model. The mod- 
ifications it ran perform include moving, adding, and 
deleting activities. 

The initial sequence which ASPEN receives from 
WITS usually contains conflicts since the specified re- 
quests from WITS may be incomplete e.g., missing 
activities, too many activities, or no designated start 
times. For example, the initial sequence in Figure 4(a) 
has only one go_to_Iocation activity which changes 
the location state variable to a value that represents 
the coordinates of the rover. Because the science re- 
quests have constraints for different values of the loca- 
tion state variable, conflicts are immediately detected. 
To resolve these conflicts, new go_to_location activi- 
ties are created with the appropriate parameter values. 
This repair algorithm is implemented at a level inde- 
pendent of the model. ASPEN computes the set of 
conflicts, and based on the type of conflict chosen for 
repair, it computes the set of possible modifications 
that might resolve the conflict. In the example above, 
the conflict is on a state variable (location), so ASPEN 
looks for an activity type, go_to_location, that can 
change the value of the state variable. 

5 Automated Path Planning 

The automated path planner system consists of 
an obstacle map generation system and a path plan- 
ner [8]. The obstacle map was generated when the 
panorama data was provided. The obstacle map was 
then subsequently used by the path planner. Obsta- 
cles were determined via segmentation of objects in 
the scene and morphological image operations were 
used to correct for errors in obstacle outlines and re- 
move noise originating from the position data. This 
will be extended in the next year to include the use 
of range maps in generating the obstacle map. The 
path planner utilizes a modified A* search algorithm 
to search through the obstacle map from the initial 
to final point. The following refinement stage elimi- 
nates extraneous via- points, resulting in very similar 
paths to the more computationally expensive A m algo- 
rithm. The resulting path is a piecewise linear path of 
via-points between waypoints. Waypoints are points 
which the rover must traverse through, e.g., locations 
where rover activities will be performed. The way- 
points might be specified by a user or by the auto- 
mated scheduler. 


Figures 2 and 4 sin >w r » *s u ! t * . displayet I in t lie WITS 
Mosaic V iew, of the automated path planner using 
panorama images from the JPL Marsyard. The circle 
icons are science targets where science activities will 
be performed. The square icons are waypoints which 
the user has specified were the rover must traverse to 
before beginning the science activities. The visualized 
paths are not. smooth due to inaccuracies in the cam- 
era models at the boundaries of neighboring images 
(the camera models are used to project the 3D path 
points into the image plane for display in the Mosaic 
View). The crosses along the path in Figure 3 are 
the via-points which the path planner automatically 
generated to specify a collision-free path. 

6 Automated Scheduling Demonstra- 
tion 

Results from a demonstration of the automated 
scheduling with distributed operations is shown in Fig- 
ures 4(a), 4(b), and 5. WITS was used to visualize 
the terrain around the rover, generate the initial sci- 
ence targets and activities, and to send the final se- 
quence to the Rocky? rover. ASPEN utilized auto- 
mated resource analysis, planning, and scheduling to 
take the initial sequence from WITS and generate a 
more complete and valid final sequence which w*as re- 
turned to WITS. The Rocky? research rover has been 
developed at JPL by the Long Range Science Rover 
task [1] of the NASA Telerobotics program. The rover 
was placed in the JPL Marsyard and a set of panorama 
images was taken by the 1.5 meter tall deployable mast 
on the rover. These images are shown in the Mosaic 
Views of Figures 4(a) and 4(bL 

The WITS user selected a dig target location (L2) 
to the West and used the Task Window to specify a 
dig activity there and set the priority for the dig to 
priority level 1. Then the user selected a spectrometer 
read target (L3) to the South and used the Task Win- 
dow to specify the spectrometer read activity and left 
the priority level at the default level 3. Then the user 
specified another dig target (L4) to the East and used 
the Target Task Window to specify the dig activity 
there with the default priority level 3. Finally, the op- 
erator selected a science image target (L5) on a rock to 
the North-East and used the Task Window to specify a 
science image task (image_navjt ime) to be performed 
there, and left the priority for the activity at the de- 
fault level 3. Using the activity pop-up window, the 
operator specified that full-illumination of the target 
was desired when taking the image and the desired lo- 
cal time for taking the image was automatically com- 



Figure 2: Path Before Path Planning 



Figure 3: Path After Path Planning 


puted and inserted in the command. The WITS client 
called a CGI program at the server computer to access 
the SPICE system to compute the ephemeris data to 
determine the angle to the sun from the target loca- 
tion for various possible times. Figure 4(a) shows the 
Panorama and Mosaic views in WITS after the op- 
erator specified the spectrometer read, two digs, and 
science image activities. 

The sequence that was generated 1 T T 
below in Terse format. 

go_to_location (rover , anytime, i y - 
dig(L2, anytime, 13^ 
spectrometer_read (L3 , anytime, 14) 
dig(L4, anytime, 15) 


image_nav_t ime(L5 , 1998-092/10: 15:00.000, 

16 , full , now, 60) 

The operator then sent the sequence to ASPEN 
by pressing the Scheduler button in the WITS Se- 
quence window. WITS sent the sequence to ASPEN 
in spacecraft activity sequence (SASF) format [9]. 
SASF is a standard spacecraft sequencing format, with 

• • • -*• r . i • » ... ti . 

*■ - 1 ..... ... ..I 

viewed hv ! lie nperaror. ASPEN received the sequence 
from W ITS and performed planning, resource analy- 
sis. and constraints checking to generate a new se- 
quence to achieve the science goals within operations 
constraints. The new sequence had the science im- 
age task first, due to its time of execution constraint, 






(a) Initial Sequence In WITS (b) Sequence Returned from ASPEN 




Figure 5: Rocky7 Rover Performing Dig Activity 


one of the digs was removed due to overall execution 
time constraint, and a dump was inserted after the dig 
due to mission rules. Also, traverse (go.toJ.ocat ion) 
commands and face location (f ace Jocation) activi- 
ties were inserted to position the rover at the activity 
locations. Straight line paths were used due to the 
lack of obstacles in the demo area. This new sequence 
was sent back to WITS in SASF format, and is shown 
below in Terse format. 

go_to_location(rover , 1998-092/08:30:00, 1) 
go_to_location(L6 , 1998-092/08:51:30, 6) 
f ace_locat ion(L7 , 1998-092/09:51:30, 7) 
image_nav_time (L5 , 1998-092/ 10:15:00, 

5, full, now, 60) 

go_to_location(L8 , 1998-092/ 10:59:00, 8) 
f ace_location(L9 , 1998-092/11:59:00, 9) 
dig(L4, 1998-092/12:13:45, 4) 
dump (L4 , 1998-092/12:23:45, 10) 
go_to_locat ion(L10 , 1998-092/12 : 54 : 00 , 11) 
f ace_location(Lll , 1998-092/13:54:00, 12) 
spectrometer_read(L3, 1998-092/14:08:50, 3) 

Figure 4(b) shows the Panorama and Mosaic views 
in WITS visualizing the updated sequence after re- 
ceiving it from ASPEN. 

The sequence that was returned from ASPEN to 
WITS was expanded into Command format and sent 
for execution on the Rocky 7 rover. The Rocky7 rover 
performing the dig activity is shown in Figure 5. 

The ASPEN system was used in this demonstra- 
tion in batch mode where it worked autonomously to 
generate an updated valid sequence. The ASPEN tool 
could also have been used interactively by an operator 
to do further interactive scheduling before providing 
the updated sequence to WITS. The ASPEN graphi- 


cal user interface for t his purpose is shown iri Figure 6. 
Also, ASPEN could have returned resource analysis 
results to the WITS operator. 

7 Conclusions 

The integration of automated planning and 
scheduling and automated path planning with a dis- 
tributed operations system, as represented by the in- 
tegrated system described in this paper, hits the po- 
tential to greatly increase the capabilities of planetary 
rover mission operations, Internet- based distributed 
operations will enable a large science team to fully par- 
ticipate in daily mission operations from their home 
institutions. Automated planning and scheduling will 
enable the distributed science team to rapidly perform 
trade-offs while generating their mission activity se- 
quences and allow them to have confidence that the 
sequence that they provide to the uplink engineering 
team is achievable. Automated path planning enables 
the automated scheduling system to determine tra- 
verse costs between science target locations. 

The system described in this paper is the first dis- 
tributed operations system for planetary rover oper- 
ations utilizing automated planning and scheduling 
and automated path planning. While the system rep- 
resents the functionality desired for future planetary 
rover operations, each component of the system will 
still need to evolve to provide the complete capabili- 
ties, with the necessary robustness, to enable the sys- 
tem to be used in the next Mars rover missions. Spe- 
cific enhancements which are currently being devel- 
oped include the following. The WITS system will in- 
tegrate more Internet security features as well as en- 
hance its distributed collaboration capabilities. The 
ASPEN system will utilize more complete activity 
models. The obstacle map that the path planner uses 
will be generated utilizing range information as well 
as image intensity information, and the path planner 
will generate paths for different risk levels, c.g., low, 
medium, a nd high risk paths. The operations team 
will then be able to also do trade-offs which include 
risk levels. 
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